Integrative encoding system and adaptive decoding system

ABSTRACT

An integrative encoding system for encoding and transmitting a plurality of video signals having different resolutions that correspond to a plurality of display units. The integrative encoding system has a compression processor, an editing processor, and an integrated services digital broadcasting (ISDB) transmitter. In a first embodiment, the compression processor performs adaptive dynamic range coding (ADRC) to compress each of said plurality of video signals on a block basis by reducing the dynamic range. In another embodiment, the compression processor performs hierarchical encoding on the plurality of video signals by selectively replacing pixels of a higher resolution level with pixels from a lower resolution level calculated by combining pixels from said higher resolution level; thereby encoding a hierarchy of resolution levels within the plurality of video signals without increasing the amount of data. An adaptive decoding system receives and decodes the transmitted plurality of video signals for display by the plurality of display units.

This is a divisional of application Ser. No. 10/160,418, filed May 31,2002 now U.S. Pat. No. 6,636,640, which is a continuation of applicationSer. No. 09/254,536 filed Mar. 9, 1999, now U.S. Pat. No. 6,480,630,issued Nov. 12, 2002, which is a 371 of PCT/JP98/03116, filed Jul. 10,1998.

FIELD OF THE ART

The present invention relates to a video encoder, video decoder, videoprocessor and methods thereof, and more particularly to such methods andapparatuses capable of efficiently encoding a picture and furtherobtaining a decoded picture in an appropriate form to a monitor fordisplaying the picture.

BACKGROUND ART

FIG. 16 shows a structural example of a video transmission system fortransmitting a picture from its transmitting side to receiving side.

On the transmitting side, in a high definition video camera 201, anobject is imaged and a high definition picture formed of 1920×1035pixels in horizontal by longitudinal in the 16:9 aspect ratio(hereinafter, it is preferably referred to as HD picture) is outputted,for example. The HD picture for approximately 30 frames will be normallyoutputted per second from the video camera 201. But here the HD pictureis subjected to interlaced scanning. Therefore, the HD picture forapproximately 60 fields is outputted per second from the video camera201.

On the transmitting side, in a standard or low definition video camera202, an object is imaged and a standard or low definition picture in the4:3 aspect ratio (hereinafter, it is preferably referred to as SDpicture) is outputted, for example. If the video camera 202 is in thenational television system committee (NTSC) system or the phasealternation by line (PAL) system, one frame is formed of 720×480 pixelsor 720×576 pixels respectively and the number of frames per second (thenumber of fields) is 30 (60) or 25 (50) respectively, for example.

On the transmitting side, in a progressive imager 203, an object isscanned and the scanned picture (hereinafter, it is preferably referredto as progressive picture) is sequentially outputted. And on thetransmitting side, in a computer 204, a picture formed of, e.g., 640×480pixels is generated and outputted as computer graphics (CG).

The ratio in horizontal to longitudinal of pixels that compose picturesto be outputted by the video cameras 201 and 202 is approximately 1:1.1.The ratio in horizontal to longitudinal of pixels that compose a pictureto be outputted by the computer 204 is 1:1.

As the above, pictures different in its aspect ratio, the number ofpixels, its scanned method, the ratio of pixels in horizontal andlongitudinal, etc., will be outputted from the video cameras 201 and202, progressive imager 203 and computer 204.

These pictures are inputted to an editor 205. In the editor 205,pictures from the video cameras 201 and 202, progressive imager 203 andcomputer 204 are edited respectively. All the edited pictures areconverted into progressive pictures formed of, e.g., 1920×1080 pixelsand outputted to a source encoder 206. In the source encoder 206, thepictures outputted from the editor 205 is coded (MPEG-coded) based onthe moving picture experts group (MPEG) standard for example, and thusobtained coded data is supplied to a channel encoder 207.

The adoption of that converting all of pictures different in the numberof pixels, its scanned method, etc., into 1920×1080-pixel progressivepictures by advanced television (ATV) has been planned.

In the channel encoder 207, channel coding is performed to improve thereliability of the coded data in transmission. That is, in the channelencoder 207, e.g., error correcting codes (ECCs) are added to the codeddata as processing for error correction, and further it is subjected toprescribed modulation or the like. The transmit data obtained by theprocessing in the channel encoder 207 is transmitted via a transmissionline 211.

On the receiving side, the transmit data transmitted from thetransmitting side as the above is received. This transmit data issupplied to a channel decoder 208 to be channel-decoded. Specifically,prescribed demodulation is performed and further error correction usingthe ECCs or the like is performed, for example.

The coded data obtained as a result of the processing in the channeldecoder 208 is supplied to a source decoder 209. In the source decoder209, the coded is expanded by, for example, decoded (MPEG-decoded) basedon the MPEG standard, and thus obtained picture data is supplied to aprocessor 210.

In the processor 210, the picture data from the source decoder 209 isprocessed to be matched to the format of an output device to output thepicture data. That is, in the case where the picture data is displayedin an HD display device 221 for displaying HD pictures, in the processor210, the picture data outputted by the source decoder 209 is processedinto an interlace-scanned HD picture composed of, e.g., 1920×1035 pixelsin the 16:9 aspect ratio. In the case where the picture data displayedin an SD display device 222 for displaying SD pictures, in the processor210, the picture data outputted by the source decoder 209 is processedinto an SD picture in the NTSC system or the PAL system, composed of720×480 pixels or 720×576 pixels for example. In the case where thepicture data is printed out by a printer 223, in the processor 210, thepicture outputted by the source decoder 209 is converted into a pictureof which the ratio of pixels in horizontal to longitudinal iscorresponded to the printer 223. On the other hand, in the case wherethe picture data is displayed on a computer display 224, in theprocessor 210, the picture data outputted by the source decoder 209 isprocessed into a picture composed of 640×480 pixels for example.

In the HD display device 221, the SD display device 222, the printer 223and the computer display 224, the picture from the processor 210 isdisplayed or printed out.

By the way, heretofore, the editing processing by the editor 205, thecompression processing by the source encoder 206 and the channel codingprocessing by the channel encoder 207 on the transmitting side havequasi conducted respectively and independently.

For example, the compressed data has less information amount than thedata before compression processing. Thus, if the compressed data can beset to be edited, a load on the editor 205 can be reduced. However, ifthe picture is MPEG-coded in the source encoder 206 as described above,the bit stream obtained as its result becomes difficult to be editedunless in a group of picture (GOP) unit, and editing of that is limitedto so-called cut editing by only connecting the GOPs. Since compressionprocessing regardless of the editing processing by the editor 205 isperformed in the source encoder 206, the compressed data cannot beedited in a frame unit and it is difficult to give various effects onit.

Moreover, for example, in the source encoder 206, compression processingis not performed in consideration of the addition of ECCs by the channelencoder 207. Therefore, for example, if ECCs are added to the coded dataobtained by the compression processing, sometimes compressibility as theentire data after the addition of ECCs has deteriorated.

As the above, heretofore, since the processing necessary for picturecoding, e.g., the editing processing, compression processing and channelcoding processing, etc., has not performed in consideration of the otherprocessing, it has been difficult to perform efficient processing.

DISCLOSURE OF INVENTION

The present invention is provided considering the above aspects, whichenable it to perform efficient processing.

A video encoder according to the present invention is characterized byincluding a processing means for performing one or plural processingnecessary to encode a picture considering the other processing.

A method for encoding a picture according to the present invention ischaracterized by performing one or more processing among the pluralprocessing necessary to encode a picture considering the otherprocessing.

A video decoder according to the present invention is characterized byincluding a generation means for generating a decoded picturecorresponding to the resolution of an output device for outputting thepicture by linearly coupling transmit data to prescribed coefficients.

A method for decoding a picture according to the present invention ischaracterized by generating a decoded picture corresponding to theresolution of an output device for outputting the picture by linearlycoupling the transmit data to the prescribed coefficients.

A video processor according to the present invention is characterized byincluding a processing means for performing one or more processing amongthe plural processing necessary to encode a picture considering theother processing, and a generation means for generating a decodedpicture corresponding to the resolution of an output device foroutputting the picture by linearly coupling the data obtained as theresult of processing by the processing means to the prescribedcoefficients.

A method for processing a picture according to the present invention ischaracterized by performing one or more processing among the pluralprocessing necessary to encode a picture considering the otherprocessing, and generating a decoded picture corresponding to theresolution of an output device for outputting the picture by linearlycoupling thus obtained data to the prescribed coefficients.

In the video encoder according to the present invention, the processingmeans performs one or more processing among the plural processingnecessary to encode the picture considering the other processing.

In the method of encoding a picture according to the present invention,one or more processing among the plural processing necessary to encodethe picture are performed considering the other processing.

In the video decoder according to the present invention, the generationmeans generates a decoded picture corresponding to the resolution of anoutput device for outputting the picture by linearly coupling transmitdata to prescribed coefficients.

In the method for decoding a picture according to the present invention,a decoded picture corresponding to the resolution of an output devicefor outputting the picture is generated by linearly coupling thetransmit data to the prescribed coefficients.

In the video processor according to the present invention, theprocessing means performs one or more processing among the pluralprocessing necessary to encode the picture considering the otherprocessing, and the generation means generates a decoded picturecorresponding to the resolution of an output device for outputting thepicture by linearly coupling the data obtained as the result ofprocessing by the processing means to the prescribed coefficients.

In the method for processing a picture according to the presentinvention, one or more processing among the plural processing necessaryto encode the picture is performed considering the other processing, anda decoded picture corresponding to the resolution of an output devicefor outputting the picture is generated by linearly coupling thusobtained data to the prescribed coefficients.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an embodiment of a video transmissionsystem in which the present invention is applied.

FIGS. 2A and 2B are diagrams explaining the ADRC processing.

FIG. 3 is a block diagram showing a structural example of ADRCprocessing circuit.

FIG. 4 is a block diagram showing a structural example of sync blockforming circuit.

FIG. 5 is a diagram showing the format of a sync block.

FIGS. 6A to 6C are diagrams explaining hierarchical coding.

FIG. 7 is a block diagram showing a structural example of hierarchicalencoding circuit.

FIG. 8 is a block diagram showing a structural example of ISDBtransmitter.

FIG. 9 is a block diagram showing a structural example of ADRC decodingcircuit.

FIG. 10 is a block diagram showing another structural example of ADRCdecoding circuit.

FIG. 11 is a block diagram showing a structural example of ISDBreceiver.

FIG. 12 is a block diagram showing a structural example of resolutioncreating circuit.

FIG. 13 is a diagram showing a classificatory block and a predictivevalue calculating block.

FIGS. 14A and 14B are diagrams explaining classification processing.

FIG. 15 is a block diagram showing a structural example of forecastingcircuit 134 of FIG. 12.

FIG. 16 is a block diagram showing a structural example of videotransmission system for transmitting a picture from its transmittingside to receiving side.

BEST MODE FOR CARRYING OUT THE INVENTION

An embodiment of the present invention will be described in detail withreference to the accompanying drawings.

FIG. 1 shows a structural example of the embodiment of videotransmission system in which the present invention is applied.

On its transmitting side, pictures composed of the maximum number ofpixels capable of being outputted by an extended definition video camera1, a video camera 2 in low definition, etc., are outputted from them.Specifically, for example, a progressive HD picture which is composed of1920×960 pixels, and of which the aspect ratio is 16:9 and the framerate is approximately 30 frame/sec, is outputted from the video camera1. And for example, a progressive SD picture which is composed of640×480 pixels, and of which the aspect ratio is 4:3 and the frame rateis approximately 30 frame/sec, or a progressive SD picture which iscomposed of 720×480 pixels, and of which the aspect ratio is the 4:3 andthe frame rate is approximately 15 frame/sec is outputted from the videocamera 2.

On its receiving side, a progressive imager 3 scans an object andoutputs for example, a progressive picture of which the longitudinalnumber of pixels is integral times of 480. And a computer 4 generatesand outputs for example, a picture which is composed of 640×480 pixelsand of which a ratio in horizontal to longitudinal is 1:1, as computergraphics (CG).

Such pictures outputted by the video cameras 1 and 2, progressive imager3 and computer 4 are supplied to an integrative encoding system 6 (theprocessing means).

In addition, for example, a progressive picture of which thelongitudinal number of pixels is integral times of 480 is supplied froma network 5, e.g., an internet or the like, to the integrative encodingsystem 6. From the network 5, a picture of which the ratio in horizontalto longitudinal of each pixel is 1:1 is supplied similarly to thecomputer 4.

Here, the video cameras 1 and 2 are made to output the pictures composedof the maximum number of pixels capable of being outputted by them,because if aiming at such pictures for the processing, decoded picturessuperior in image quality can be generally obtained comparing with thecase of aiming at pictures having less number of pixels in one picture,e.g., pictures in an interlace system.

Moreover, all the longitudinal number of pixels of the pictures to besupplied to the integrative encoding system 6 is the prescribed value,i.e., integral times of 480 here for example. And their frame rate isalso the prescribed value, integral times of 15. This is because when anSD picture is generated by thinning out the pixels forming an HD picturein the spatial direction or temporal direction or when an HD picture isgenerated by interpolating the pixels forming an SD picture in thespatial direction or temporal direction, the deterioration in the imagequality of the SD picture or HD picture to be generated can be reduced.

The longitudinal number of pixels of the HD picture to be outputted tothe video camera 1 is set as 960 pixels, because when the longitudinalnumber of pixels is integral times of 480, “960” most approximates to“1035” being the longitudinal number of pixels of existing HD pictures(FIG. 16), and thus, the deterioration in its image quality can bereduced when in performing crossing over.

The longitudinal number of pixels of the SD picture to be outputted tothe video camera 2 is set as 480 pixels, because that value is theintegral number of 480 most approximate to the number of pixels adoptedin the present NTSC system or PAL system or the like.

If the horizontal number of pixels of the SD picture of which the aspectratio is 4:3 to be outputted to the video camera 2 is set to 640, theratio in horizontal to longitudinal of the pixel becomes 1:1(=4×480:3×640). As a result, compatibility with the picture to beoutputted by the computer 4 or the picture to be supplied from thenetwork 5 is apt to be obtained.

If the horizontal number of pixels of the SD picture of which the aspectratio is 4:3 to be outputted to the video camera 2 is set to 720, theratio in horizontal to longitudinal of the pixel becomes 8:9(=4×480:3×720). This is equal to the ratio in horizontal to vertical(8:9=16×960:9×1920) of the pixels that form the HD picture to beoutputted by the video camera 1. Therefore, in this case, if thehorizontal and vertical numbers of pixels of the SD picture to beoutputted by the video camera 2 are doubled and the longitudinal numberof pixels is set to 960 pixels that is equal to the longitudinal numberof pixels of the HD picture to be outputted by the video camera 1, quasiroundness can be kept (that is, the pictures can avoid being long in thehorizontal direction or the vertical direction).

In the integrative encoding system 6, one or more processing necessaryfor its encoding, e.g., editing processing, compression pressing,channel coding processing or the like, are performed on pictures(digital video signals) supplied thereto considering the otherprocessing. Here, the editing processing includes not only quasi videoediting processing in which cut editing is performed or effects aregiven but also adding information to realize an integrated servicesdigital broadcasting (ISDB: it is interactive broadcasting thatdigitalizes and transmits various information) and information to attachother values added (for example, information necessary to obtain adecoded picture in further high image quality), and linking a certainpicture to another picture, and the like, for example.

Transmit data obtained by the processing in the integrative encodingsystem 6 is transmitted to the receiving side via a transmission line13. As the transmission line 13, other than communication lines such asa satellite line, terrestrial wave, CATV network, public network,internet, etc., the process of magnetic recording/reproducing and alsorecording mediums such as a magnetic disk, optical disk, magnetic tape,magneto-optical disk, and other recording mediums are included.

The transmit data transmitted via the transmission line 13 is receivedon the receiving side and supplied to an adaptive decoding system 7 (thegeneration means). The adaptive decoding system 7 is connected to outputdevices for outputting a picture, e.g., a high definition display unit 9for displaying the HD picture, a display unit 10 in standard definitionor low definition for displaying the SD picture, a printer 11 forprinting out a picture, and a computer display 12 connected to acomputer (however, the printer 11 is connected via a converter foraspect ratio of pixel 8). The adaptive decoding system 7 generatesdecoded pictures corresponding to the respective resolution of thedisplay units 9 and 10, printer 11 and computer display 12 andoutputting these pictures to them respectively, by linearly coupling thetransmit data to the prescribed coefficients.

In the display units 9 and 10 and the computer display 12, the decodedpicture from the adaptive decoding system 7 is displayed, and in theprinter 11 the decoded picture from the adaptive decoding system 7 isprinted out.

Note that, the difference between the ratio in horizontal tolongitudinal of each pixel in the display units 9 and 10 and thecomputer display 12 (hereinafter, it is preferably referred to as aspectratio of pixel) and the aspect ratio of pixel forming the decodedpicture is absorbed by horizontal scanning in each of the display units9 and 10 and the computer display 12.

On the other hand, in the printer 11, since the difference of the aspectratio of pixel cannot be absorbed by such horizontal scanning, theconverter for aspect ratio of pixel 8 is provided in the precedingstage. There the aspect ratio of pixel of the decoded picture isconverted into a value applicable to the printer 11.

Then, processing in the integrative encoding system 6 will be described.

In the integrative encoding system 6, for example, the compressionprocessing is performed considering the editing processing.

In the integrative encoding system 6, an adaptive dynamic range coding(ADRC) processing is performed as the compression processing, forexample.

Here, the ADRC processing will be described briefly.

To simplify the description, if considering a block composed of fourpixels in line, in the ADRC processing, the maximum value MAX and theminimum value MIN of their pixel values are detected as shown in FIG.2A. And DR=MAX−MIN is set as the local dynamic range of the block, andthe pixel values of the pixels forming the block are requantized into Kbit.

Specifically, the minimum value MIN is subtracted from each pixel valuein the block. The subtracted values are divided into DR/2^(K). And thepixel values are converted into codes (ADRC codes) corresponding to thusobtained divided values. More specifically, if assuming K=2 for example,it is determined that the divided values belong to which range amongranges obtained by dividing the dynamic range DR into four (=2²) asshown in FIG. 2B. If the divided value belongs to the range of, e.g.,the lowest level, the second level from low, the third level from low,or the top level, it is coded into two bits, e.g., 00B, 01B, 10B or 11B(B shows a binary number).

In the ADRC, a smaller value than the number of bits assigned to thepixel is used as the number of bits K at the time of requantizing.Therefore, each pixel is compressed into such small number of bits(however, the minimum value MIN and the dynamic range DR are generatedfor each block other than that).

Note that, its decoding can be performed by converting the ADRC code00B, 01B, 10B or 11B into, e.g., the central value L₀₀ of the lowestlevel range obtained by dividing the dynamic range DR into four, thecentral value L₀₁ of the second level range from low, the central valueL₁₀ of the third level range from low or the central value L₁₁ of thetop level range, and adding the minimum value MIN to the value.

FIG. 3 shows a structural example of ADRC processing circuit thatperforms the ADRC processing.

Video data is supplied to a blocking circuit 21 and divided there intoblocks in prescribed size. That is, the blocking circuit 21 divides thevideo data into blocks of, e.g., horizontal 4 by longitudinal 4 pixels,and supplies these blocks to a minimum value detecting circuit 22, amaximum value detecting circuit 23 and a computing unit 25.

In the minimum value detecting circuit 22, the minimum value MIN isdetected from among 16 (=4×4) pixels forming the block from the blockingcircuit 21. This minimum value MIN is outputted as one of the signalsobtained by the ADRC processing as well as supplied to a computing unit24 and the computing unit 25.

At the same time, in the maximum value detecting circuit 23, the maximumvalue MAX is detected from among 16 pixels forming the block from theblocking circuit 21 and supplied to the computing unit 24.

In the computing unit 24, the minimum value MIN is subtracted from themaximum value MAX to obtain the dynamic range DR of the block. Thisdynamic range DR is outputted as one of the signals obtained by the ADRCprocessing as well as supplied to a quantizing circuit 26.

In the computing unit 25, the minimum value MIN of the block issubtracted from each of the 16 pixels forming the block, and thesubtracted values are supplied to the quantizing circuit 26. In thequantizing circuit 26, the subtracted values from the computing unit 25are quantized in a quantization step corresponding to the dynamic rangeDR from the computing unit 24. That is, in the quantizing circuit 26,for example, the output of the computing unit 25 is divided intoDR/2^(K), and a value which the decimal fractions has omitted isoutputted as an ADRC code CODE (requantized result of pixel).

In this embodiment, the number of bits K in requantizing is determined,for example, corresponding to the size of the dynamic range DR of eachblock and the ADRC code is variable length.

For example, it is assumed that four threshold values T1, T2, T3 and T4are now set and their size relationship is 0<T1<T2<T3<T4 and also eightbits are assigned to the pixels of an original picture (thus, T4 is lessthan 2⁸).

In this case, in the quantizing circuit 26, it is determined whether ornot the dynamic range DR is within one of 0 or more and less than T1, T1or more and less than T2, T2 or more and less than T3, T3 or more andless than T4 and T4 or more and less than 2⁸. And if the dynamic rangeDR is within one of those, for example, 0 through 4 bits arerespectively assigned as the number of bits K in requantizing.Therefore, in this case, the ADRC code becomes 0 bit at the minimum and4 bit at the maximum.

When the ADRC code is variable length, the number of bits K of the ADRCcode is necessary for its decoding. Therefore, the quantizing circuit 26outputs a threshold code showing that the dynamic range DR is withinwhich range among the above ranges. This threshold code is outputted asthe result of ADRC processing together with the minimum value MIN,dynamic range DR and ADRC code.

Note that, the minimum value MIN, dynamic range DR and threshold codeare set in variable length for example.

In the above case, the ADRC code is set in variable length, however, theADRC code can be set in fixed length provided that the number of bits Kin requantizing is set to a fixed value irrespective of the dynamicrange DR of the block.

The ADRC code obtained by the above ADRC processing becomes less thanthe number of bits assigned to the original pixels. On the other hand,the minimum value MIN, dynamic range DR and ADRC code obtained by theADRC processing can be used in a block unit, thus the ADRC-processedpicture can be edited in a frame unit for example.

From the above, by conducting the ADRC processing as the compressionprocessing and setting the result of the ADRC processing as an object ofvarious editing processing, the almost same editing processing as thecase where the original picture before ADRC processing is set an objectcan be performed, and at the same time, a load of the processing can bereduced comparing with the case where the original picture before ADRCprocessing is set as the object.

Thus it can be said that the ADRC processing as compression processingis performed considering the editing processing, and the editingprocessing can be executed efficiently.

The detail of the ADRC has been disclosed in the patent laid-open No.Hei3(1991)-53778 previously filed by the present applicant, or the like,for example.

The ADRC can be referred to as block coding since it performs coding ina block unit. In addition to the ADRC, however, the block codingincludes coding which obtains the mean value and the standard deviationof pixels forming a block and also 1-bit flag showing the sizerelationship between each pixel and the mean value, and the like. Suchblock coding may be adopted as the compression processing by theintegrative encoding system 6.

In the above case, the minimum value MIN and the dynamic rang DR areincluded in the result of the ADRC processing. In the result of theADRC, also, the minimum value MIN and the maximum value MAX of theblock, or the dynamic range DR and the maximum value MAX of the blockcan be included in addition to that.

Moreover, in the above case, a block is composed of one frame ofhorizontal 4 pixel by longitudinal 4 pixel. However, the block can becomposed of pixels forming plural frames that temporary continue.

By the way, as a method for transmitting the minimum value MIN, dynamicrange DR, threshold code and ADRC code for each block obtained by theADRC processing, for example, there is a method which forms a blockdisposes a prescribed amount of data of the ADRC result (hereinafter, itis preferably referred to as sync block) following a synchronizationpattern for matching and performing transmission in such sync blockunit.

When transmission is performed in a sync block unit, the minimum valueMIN, dynamic range DR and threshold code are fixed length as the above.Therefore, if these data are disposed at fixed positions in a syncblock, even if one sync block cannot be obtained by failure, it does notaffect minimum values MIN, dynamic ranges DR and threshold codesdisposed in other sync blocks.

Since the ADRC code is variable length, however, in the case where anADRC code is separately disposed in plural sync blocks because it cannotbe contained in one sync block for example, the failure of the one syncblock sometimes affects the other sync blocks. Specifically, if thefirst sync block has failed among the plural sync blocks, it is unknownthat the ADRC code disposed at the top of the second sync block iscorresponding to which position's pixel in the block, further it isunknown that the bit disposed as the ADRC code is whether a bit formingthe following (a part of) ADRC code disposed at the end of the firstsync block or the first bit of the ADRC code. As a result, even an ADRCcode disposed in a sync block after the second cannot be fetched. As theabove, a certain sync block error propagates to the other sync blocks.

By the way, even if all the ADRC codes of a certain block has lost, ifthe minimum value MIN has known, the block in which all the pixels havethe minimum value MIN as a pixel value can be reproduced. However, sincethis block has the same pixel value, practically flat, thereproducibility of the original picture is low.

To realize its further higher reproducibility, it can be considered thatthe ADRC code is separated into, for example, the most significant bit(MSB) and others (hereinafter, it is preferably referred to as remainingbits) and disposing also the MSB at a fixed position in the sync blocksimilarly to the minimum value MIN, dynamic range DR and threshold code.In this case, even if the remaining bit has lost, the block formed ofbinary can be obtained by inversely quantizing the MSB based on thedynamic rang DR. Thus, the picture with higher reproducibility can beobtained comparing with the case where all the ADRC codes have lost.

FIG. 4 shows a structural example of sync block forming circuit thatperforms sync block processing to form the above sync block.

The minimum value MIN, dynamic range DR and threshold code outputtedfrom the ADRC processing circuit (FIG. 3) are supplied to a multiplexer32 and the ADRC code CODE is supplied to a separator 31. In theseparator 31, the ADRC code is separated into the MSB and the remainingbits and both are supplied to the multiplexer 32.

The synchronization pattern has supplied to the multiplexer 32 inaddition to the above data. The multiplexer 32 performs time divisionmultiplexing on the data supplied thereto and forms and outputs a syncblock such as shown in FIG. 5.

That is, as shown in FIG. 5, the synchronization pattern having a fixedlength is disposed at the top of the sync block, and added data having afixed length is disposed following it. Here, the added data is formedonly fixed length data such as the threshold code or the like. Followingthe added data, the remaining bits are disposed by a prescribed numberof bytes, that is, after the added data, the remaining bits are disposedfrom the head to an N₁-1-th byte.

The dynamic range DR, MSB and minimum value MIN are disposed after fromthe top to an N₁-th byte in the order of DR, MSB, MIN, MSB, DR, . . . ,for example. Following the prescribed number of the disposed dynamicrange DR, MSB and minimum value MIN, the remaining bits are disposedagain. Then, after an N₂-th byte from the top, the prescribed number ofthe dynamic range DR, MSB and minimum value MIN are disposed again inthe aforementioned order. Hereinafter, the similar disposition isrepeated until the end of the sync block.

Since the dynamic range DR, the MSB and the minimum value MIN aredisposed from the determined position as from the N₁-th byte , N₂-thbyte, . . . from the head of the sync block as the above and these dataare fixed length, they are disposed at the fixed position in the syncblock.

In the above sync block processing, even if an error is caused in theADRC code (remaining bits), a decoded picture comparatively close to theoriginal picture can be obtained. Furthermore, even if no bit remains, adecoded picture in high reproducibility can be obtained. Thus, extremelyto say, it is unnecessary to add an ECC for error correction to theremaining bits, for example. In this case, a load against channel codingprocessing can be reduced. For this reason, it can be said that the syncblock processing is performed considering channel coding processing.

Note that, the detail of the sync block processing has been disclosed inthe patent laid-open No. 2(1990)-162980 previously filed by the presentapplicant or the like.

The integrative encoding system 6 can also perform hierarchical codingprocessing as the compression processing in place of the ADRCprocessing, for example.

In the hierarchical coding, for example, high resolution picture data isset as the picture data of the lowest hierarchy or the first hierarchy,and the picture data of the second hierarchy (compressed picture) ofwhich the number of pixels is less than the first hierarchy's isgenerated and the picture data of the third hierarchy of which thenumber of pixels is less than the second hierarchy's is generated.Similarly, picture data are generated until the most upper hierarchy.The picture data of each hierarchy is displayed on a monitor having aresolution (the number of pixels) corresponding to the hierarchy. Thus,on a user side, a picture of the same contents can be viewed byselecting picture data corresponding to the resolution of his monitorfrom among the hierarchically-coded picture data.

By the way, in the case where picture data having a certain resolutionis set as the picture data of the lowest hierarchy (first hierarchy) andthe picture data of an upper hierarchy is sequentially formed and all ofthem are stored or transmitted as they are, much storage capacity ortransmission capacity is required by the picture data of the upperhierarchy comparing with the case of storing only the picture data ofthe lowest hierarchy.

For that reason, here, a hierarchical coding without such increasing ofstorage capacity or the like is adopted as the compression processing inthe integrative encoding system 6.

For example, now the mean value of four pixels, 2×2 (horizontal byvertical) pixels in a lower hierarchy is set as the pixels (pixel value)of an upper hierarchy, and performing hierarchical coding in threehierarchies. In this case, considering 8×8 pixel as the picture of thelowest hierarchy as shown in FIG. 6A, the mean value m0 of four pixelsh00, h01, h02 and h03 at the upper left of the second hierarchy isoperated and set as the one pixel of the upper left of the secondhierarchy. Similarly, the mean value m1 of four pixels h10, h11, h12 andh13 at the upper right of the picture of the lowest hierarchy, the meanvalue m2 of four pixels h20, h21, h22 and h23 at the lower left and themean value m3 of four pixels h30, h31, h32 and h33 at the lower rightare operated and set as the one pixel at the upper right, lower left andlower right of the second hierarchy respectively. Furthermore, the meanvalue (q) of 2×2 pixels of the second hierarchy, the four pixels m0, m1,m2 and m3, is operated and set as the pixel of the picture of the thirdhierarchy, i.e., the most upper hierarchy, here.

If all the above pixels h00 to h03, h10 to h13, h20 to h23, h30 to h33,m0 to m3 and (q) are stored as they are, much storage capacity or thelike is required by the pixels m0 to m3 and (q) as described above.

Then, as shown in FIG. 6B, the pixel (q) of the third hierarchy isdisposed at, e.g., the position of the pixel m3 at the lower right amongthe pixels m0 to m3 of the second hierarchy. Thus, the second hierarchyis composed of the pixels m0 to m2 and (q).

As shown in FIG. 6C, the pixel m0 of the second hierarchy is disposedat, e.g., the position of the pixel h03 at the lower right, among thepixels h00 to h03 of the third hierarchy that has used to obtain m0.Similarly, the remaining pixels m1, m2 and (q) of the second hierarchyare disposed in place of the pixels h13, h23 and h33 of the firsthierarchy. Note that, the pixel (q) has not obtained directly from thepixels h30 to h33 but since it has been disposed in the second hierarchyin place of m3 obtained directly from them, the pixel (q) is disposedinstead of disposing the pixel m3 at the position of the pixel h33.

By conducting as the above, as shown in FIG. 6C, the total of pixelsbecomes 16 pixel of 4×4. This is the same as the case of only the pixelsof the lowest hierarchy shown in FIG. 6A. Therefore, in this case, anincrease of storage capacity or the like can be prevented.

In this connection, the decoding of the pixel m3 changed into the pixel(q) and the pixels h03, h13, h23 and h33 changed into the pixels m0 tom3 respectively can be performed as follows.

Since (q) is the mean value of m0 to m3, the equation q=(m0+m1+m2+m3)/4is satisfied. Thus, m3 can be obtained by the equationm3=4×q−(m0+m1+m2).

Since m0 is the mean value of h00 to h03, the equationm0=(h00+h01+h02+h03)/4 is satisfied. Thus, h03 can be obtained by thequation h03=4×m0−(h00+h01+h02). In similar manner, h13, h23 and h33 canbe obtained.

FIG. 7 shows a structural example of hierarchical coding circuit thatperforms the above hierarchical coding processing. In this hierarchicalcoding circuit, the aforementioned hierarchical coding in threehierarchies is performed for example.

The first hierarchy (lowest hierarchy) picture data (here, it isprogressive picture as described above) is supplied to a mean valuecalculating circuit 41 and a pixel extracting circuit 43.

In the mean value calculating circuit 41, with respect to the firsthierarchy picture, for example, the mean value of 2×2 pixels the totalfour pixels as the above is computed, and the second hierarchy pictureis generated. This second hierarchy picture is supplied to a mean valuecalculating circuit 42 and a pixel extracting circuit 44.

In the mean value calculating circuit 42, with respect to the secondhierarchy picture, for example, the mean value of 2×2 pixels the totalfour pixels is computed, and the third hierarchy picture is generated.This third hierarchy picture is supplied to a pixel inserting circuit45.

In the pixel extracting circuit 43, pixels which correspond to thepixels h03, h13 and h23 described in FIG. 6 are extracted from the firsthierarchy picture and the remains are supplied to the pixel insertingcircuit 45. In the pixel extracting circuit 44, pixels which correspondto the pixel m3 described in FIG. 6 is extracted and the remains aresupplied to the pixel inserting circuit 45.

In the pixel inserting circuit 45, the pixels of the second hierarchypicture (e.g., the pixels m0 to m2) from the pixel extracting circuit 44are inserted to positions corresponding to the pixels h03, h13 and h23of the first hierarchy picture from the pixel extracting circuit 43, andthe pixel of the third hierarchy picture (e.g., pixel (q)) from the meanvalue calculating circuit 42 is inserted to a position corresponding tothe pixel h33 of the first hierarchy picture. In the above manner, thepicture data described in FIG. 6C is formed and this is outputted as theresult of the hierarchical coding.

If according to normal hierarchical coding, much storage capacity ortransmission capacity is required by the picture data of upperhierarchies, however, according to the hierarchical coding described inFIGS. 6 and 7 (hereinafter, it is preferably referred to as improvedhierarchical coding), the amount of data to be obtained as the result isthe same as the picture of the lowest hierarchy. For this reason, theimproved hierarchical coding can be said to be information compressingprocessing.

When the hierarchical coding is performed, the picture of a lowerhierarchy can be obtained by performing, for example, interpolation orthe like using an upper hierarchy picture (however, thus obtainedpicture is not the same picture as the lower hierarchy picture but apicture deteriorated in image quality.) Therefore, even in the worst,the pictures of all the hierarchies can be obtained provided that thepicture of the most upper hierarchy can be restored, so that, forexample, the addition of an ECC for error correction is sufficient onlyperforming to the picture of the most upper hierarchy and it isunnecessary to perform to the pictures of all the hierarchies. In thiscase a load to channel coding processing can be reduced. For thisreason, it can be said that the hierarchical coding processing isperformed considering the channel coding.

Note that, in the above case, the upper hierarchy picture is generatedwith reducing the number of pixels in the spatial direction, but theupper hierarchy picture can be generated with reducing the number ofpixels in the temporal direction for example.

Then, it will be described about the addition of information forrealizing an ISDB, one of the editing processing in the integrativeencoding system 6.

FIG. 8 shows a structural example of ISDB transmitter being a part ofthe integrative encoding system 6 that realizes the ISDB.

For example, an SD picture and audio attached thereto are inputted to anencoding part 51, and they are subjected there to the compressionprocessing as the aforementioned ADRC processing. Thus obtained signalis outputted to a multiplexing part 57. Furthermore, the encoding part51 outputs a synchronizing signal which represents the timing of thecompression processing to a time code generating part 52. The time codegenerating part 52 generates a time code or the like as additiveinformation to be added to the output of the encoding part 51synchronizing with the synchronizing signal from the encoding part 51and outputs this to the multiplexing part 57.

Also in an encoding part 53 or a time code generating part 54, theprocessing similar to the encoding part 51 or the time code generatingpart 52, except for that an object of the processing is not SD picturebut HD picture, is performed respectively. Both of the coded dataobtained by the compression processing in the encoding part 51 and thetime code outputted by the time code generating part 54 are supplied tothe multiplexing part 57.

To an encoding part 55, for example, a computer program, data necessaryto execute the program, facsimile data and also data for realizingmultimedia such as local information are inputted. These data arecompressed there and outputted to the multiplexing part 57 as codeddata. Furthermore, the encoding part 55 outputs the synchronizing signalwhich represents the timing of the compression processing to an additiveinformation generating part 56. The additive information generating part56 generates additive information which represents a type of the datacompressed in the encoding part 55, synchronizing with the synchronizingsignal from the encoding part 55 and outputs this to the multiplexingpart 57.

Here, the local information to be inputted to the encoding part 55 isinformation peculiar to each area, and it includes, for example, aweather forecast, map, information on establishments (e.g., the servicecontents and business hours of restaurants, etc.) and advertisements ineach area. With respect to such local information, the additiveinformation generating part 56 generates an area code which representsan area corresponding to each local information as added information.

In the multiplexing part 57, the outputs of the encoding part 51, timecode generating part 52, encoding part 53, time code generating part 54,encoding part 55 and additive information generating part 56 aremultiplexed and outputted.

Then, FIG. 9 shows a structural example of the adaptive decoding system7 (FIG. 1) in the case where the ADRC processing is performed as thecompression processing in the integrative encoding system 6. That is,FIG. 9 shows a structural example of ADRC decoding circuit being a partof the adaptive decoding system 7 that performs ADRC decoding processingwhen the result of the ADRC processing is decoded.

To a demultiplexer 101, a bit stream in which the result of the ADRCprocessing is disposed is inputted as transmit data transmitted via thetransmission line 13 (FIG. 1), and there the minimum value MIN, dynamicrange DR and ADRC code are separated from the transmit data. Note that,in the demultiplexer 101, the separation of the ADRC code is performedby separating the threshold code from the transmit data and recognizingthe number of assigned bits to the ADRC code (the above K) based on thethreshold code.

The minimum value MIN is supplied to an arithmetic unit 103, and thedynamic range DR and the ADRC code are supplied to an inverse quantizingcircuit 102, respectively. In the inverse quantizing circuit 102, theADRC code is inversely quantized in a quantization step corresponding tothe dynamic range DR, and thus obtained inversely-quantized value issupplied to the arithmetic unit 103. In the arithmetic unit 103, theinversely-quantized value from the inverse quantizing circuit 102 isadded to the minimum value MIN. Thereby, the pixels are decoded.

If obtaining a pixel for one block, the arithmetic unit 103 suppliesthis to a frame forming circuit 104. The frame forming circuit 104sequentially stores the pixels which have supplied in a block unit andoutputting them every storing of it.

In the case where the ADRC code has separated into the MSB and theremaining bits as described above, the demultiplexer 101 also performsprocessing for storing the original ADRC code by combining the MSB withthe remaining bits. Furthermore, in the case where the ADRC code hasseparated into the MSB and the remaining bits, if an error has occurredin the remaining bits, the demultiplexer 101 outputs the MSB to theinverse quantizing circuit 102 as the ADRC code.

By the way, in the ADRC decoding, even if the remaining bits are inerror, if the MSB, minimum value MIN and dynamic range DR exist, adecoded picture having fine reproducibility in a certain degree (decodedpicture similar to the original picture) can be obtained, as describedabove. However, if the minimum value MIN or the dynamic range DR is inerror, it becomes difficult to decode the block.

FIG. 10 shows a structural example of ADRC decoding circuit capable ofdecoding a block with relatively fine precision even if the minimumvalue MIN or the dynamic range DR is in error. Note that, in FIG. 10,the same reference numerals are added to corresponding parts of FIG. 9,and its description will be omitted hereinafter. That is, this ADRCdecoding circuit is basically similarly configured to FIG. 9 except thatselectors 105 and 106, a memory 107 and a restoring circuit 108 arenewly provided.

To the selector 105, the dynamic range DR outputted by the demultiplexer101 and a predicted value of the dynamic range DR′ outputted by therestoring circuit 108 are supplied. To the selector 106, the minimumvalue MIN outputted by the demultiplexer 101 and a predicted value ofthe minimum value MIN′ outputted by the restoring circuit 108 aresupplied. The demultiplexer 101 here detects whether an error isoccurred in the minimum value MIN and the dynamic range DR included inthe transmit data, and if an error is caused, it outputs an error signalto the selectors 105 and 106.

The selector 105, when not receiving the error signal, that is, no erroris caused in the minimum value MIN and the dynamic range DR, selects thedynamic range DR outputted by the demultiplexer 101 and outputs this tothe inverse quantizing circuit 102. Similarly, the selector 106, whennot receiving the error signal, selects the minimum value MIN outputtedby the demultiplexer 101 and outputs this to the arithmetic unit 103.

Therefore, in this case, the ADRC decoding processing is performedsimilar to the case of FIG. 9.

On the other hand, the decoded values of the pixels outputted by thearithmetic unit 103 are supplied to not only the frame forming circuit104 but also the memory 107. In the memory 107, the decoded values ofthe pixels from the arithmetic unit 103 are stored in each correspondingaddress.

Then, in the restoring circuit 108, the decoded values of pixels in thecircumference of a block now being an object of the ADRC decodingprocessing are read out from the memory 107 by the same number as thenumber of pixels forming a block, that is, 16 pieces in this embodimentas described above. Furthermore, the restoring circuit 108 detects theminimum value and the dynamic range (difference between the maximumvalue and the minimum value) of that sixteen pixels, and outputs each ofthem to the selectors 106 and 105 respectively as the predicted value ofthe minimum value MIN′ and the predicted value DR′ of the dynamic rangeof the block now being the object of the ADRC decoding.

The selector 105 or 106, if receiving the error signal from thedemultiplexer 101, that is, an error is caused in the minimum value MINor the dynamic range DR, selects the predicted value of the dynamicrange DR′ or the predicted value of the minimum value MIN′ from therestoring circuit 108, and outputs them to the inverse quantizingcircuit 102 or the arithmetic unit 103 respectively.

Thus, in this case, in the inverse quantizing circuit 102, inversequantization is performed using the predicted value of the dynamic rangeDR′, and in the arithmetic unit 103, the pixels are decoded using thepredicted value of the minimum value MIN′.

Marking a certain block, normally a large relationship exists betweenpixels forming the marked block and pixels in the circumference of themarked block. Therefore, according to the pixels having suchcorrelation, the dynamic range and the minimum value of the marked blockcan be predicted in relatively fine precision. As a result, using thuspredicted values enable to obtain a decoded picture almost similar tothe case of using the true minimum value MIN and dynamic range DR.

The detail of the above ADRC decoding processing has been disclosed inthe patent laid-open No. S63(1988)-257390 previously filed by thepresent applicant, for example.

When the ADRC processing is performed, even if an error has caused inthe minimum value MIN or the dynamic range DR a decoded picture in acertain degree can be obtained as described above. Moreover, byperforming sync block processing in addition to the ADRC processing, tocope with an error in the remaining bits is enabled as described above.Furthermore, also in the case of performing hierarchical coding, it issufficient to perform the processing for error correction to the pictureof the most upper hierarchy at least as described above, and it is notrequired to always perform the processing to the pictures of all thehierarchies.

For this reason, the ADRC processing, sync block processing andhierarchical coding processing can be said that quasi robust processingstrongly resistant to errors. Now if such robust processing havingresistance to errors is assumed as robust coding, the processingperformed in the integrative encoding system 6 can be said integrativecoding processing in which such robust coding and editing processing andthe like are integrated.

Among the robust coding, for example, in the ADRC processing, an amountof information is reduced and also a resistance to errors is improved byits execution. Therefore it can be said that in the ADRC processing, thecompression processing of pictures and the processing for errorcorrection are performed by organically combined.

Then, FIG. 11 shows a structural example of ISDB receiver being a partof the adaptive decoding system 7 that realizes an ISDB.

To a signal separating part 111, the transmit data transmitted via thetransmission line 13 is inputted, and there coded data obtained bycoding video (SD picture or HD picture) and audio accompanying with thatfor example, are extracted from the transmit data. The coded datagenerated in the signal separating part 111 are outputted to a decodingpart 112 respectively.

In the decoding part 112, the coded data from the signal separating part111 are decoded. That is, ADRC decoding or the like is performed on thecoded video data for example. The video and the voice accompanying withthat obtained by the decoding processing in the decoding part 112 areoutputted via a selector 113. The picture is supplied to the displaydevice 9 or 10 or the computer display 12 and displayed, or supplied tothe printer 11 via the converter for aspect ratio of pixel 8 and printedout. On the other hand, the audio is supplied to a speaker (not shown)and outputted.

In the signal separating part 111, the local information and the areacode as added information corresponding to the local information areextracted and supplied to a decoding part 114. In the decoding part 114,the local information is decoded corresponding to the area code.

That is, since it is not limited that the received local information isthe information on an area desired by the user, only the localinformation inputted accompanied with the area code corresponding to thearea previously inputted is decoded in the decoding part 114.

Inputting an area is able by operating a control part 123 for example.The inputted area information is supplied to the decoding part 114 viaan OR gate 122.

Furthermore, the area input is also able using a global positioningsystem (GPS) system (“system” means that plural devices are logicallyintegrated and it is not referred to whether or not each of deviceshaving individual configuration are contained in the same housing). Thatis, a radio wave from a GPS satellite is received by an antenna 118, andthe received signal is supplied to a GPS receiving part 119. The GPSreceiving part 119 calculates the position where the ISDB receiver ofFIG. 11 has been installed from the received signal from the antenna118, and supplies thus obtained position information (for example, itslongitude and latitude or the like) to an area discriminating part 120.The area discriminating part 120 discriminates the area from theposition information from the GPS receiving part 119, and outputs a codeassigned to that area to a memory 121 for storing. The code stored inthe memory 121 is supplied to the decoding part 114 via the OR gate 122.In the decoding part 114, only the local information inputtedaccompanied with the area code agree with that code is decoded.

The local information decoded in the decoding part 114 is supplied to amemory 115 and stored therein. Thus, in the memory 115, only the localinformation for the area where the user lives in, for example.

When viewing the local information stored in the memory 115, the useroperates so the control part 123. The local information is read out fromthe memory 115 corresponding to the operation of the control part 123and supplied to a selector 116. In the selector 116, one or all of thelocal information from the memory 115 is selected and outputted via theselector 113. Thus, the local information is displayed on the displaydevice 9 or printed out similarly to the above picture.

Note that, even the local information of the same area, informationnecessary for the user and not so are mixed therein. In the selector116, only the local information needed by the user is selectedcorresponding to the operation of the control part 123.

In the selector 113, either the output of the decoding part 112 or theoutput of the selector 116 is selected and outputted.

Furthermore, in the ISDB receiver of FIG. 8, a time code is multiplexedas added information of video and audio accompanying with it, however,as the added information of video and audio accompanying with that, forexample, an identification code to recognize a program composed of videoand audio can be multiplexed other than that. In this case, a programcorresponding to that identification code can be selected by previouslyinputting the identification code of a desired program. It enables, onlywhen a desired program has transmitted, the user to select and outputthe program.

Note that, the details of the above ISDB transmitter and ISDB receiverhave been disclosed in the patent application Nos. H7(1995)-207158 andH7 (1995)-243453 previously filed in the present applicant, for example.

Then, the adaptive decoding system 7, if the resolution of the decodedpicture is lower than the resolution of the output device that willoutput the decoded picture among the display devices 9 and 10, printer11 and computer display 12, performs resolution creating processing togenerate the decoded picture corresponding to the resolution of theoutput device by linearly coupling the decoded picture having lowresolution to the prescribed coefficients.

Note that, in the case where the number of pixels of the decoded pictureis less than the number of pixels of the output device, for example,there is a method for matching the number of pixels of the decodedpicture with the number of pixels of the output device by performing aninterpolation by means of an interpolation filter or the like. However,since such simple interpolation cannot represent high frequencycomponents not included in the original decoded picture, it cannotimprove the resolution. On the contrary, in the resolution creatingprocessing, such high frequency components can be represented asdescribed later.

Moreover, in the case where the number of pixels of the decoded pictureis more than the number of pixels of the output device, the number ofpixels of the decoded picture is reduced to coincide the number ofpixels of the output device by thinning or rearranging the mean valuesof several pixels or the like as in the case of hierarchical coding.

FIG. 12 shows a structural example of resolution creating circuit beinga part of the adaptive decoding system 7 that performs the resolutioncreating processing.

Also here, pictures having high resolution (of which the number ofpixels is large) are defined as the lower hierarchy pictures andpictures having lower resolution (or which the number of pixels issmall) are defined as the upper hierarchy pictures similarly to the caseof hierarchical coding.

For example, in FIG. 13, if the parts shown by a dot are assumed aspixels forming the lower hierarchy picture (hereinafter, it ispreferably referred to as lower pixel) and the parts shown by a circleare assumed as pixels forming the upper hierarchy picture (hereinafter,it is preferably referred to as upper pixel), the resolution creatingcircuit converts the upper hierarchy picture formed of the pixels shownby the circles into the lower hierarchy picture formed of the pixelsshown by dots.

That is, the upper hierarchy picture is supplied to a classificatoryblocking circuit 131 and a blocking circuit for calculation ofpredictive value 133.

The classificatory blocking circuit 131 forms classificatory blocks thatinclude the prescribed marked pixel from the upper hierarchy picturesupplied thereto. More specifically, the classificatory blocking circuit131 forms classificatory blocks each which is formed of 5×5 (horizontalby longitudinal) upper pixels and has the marked pixel in its center asshown by enclosing with a solid line in FIG. 13, for example.

Hereinafter, the 5×5-upper pixels which form the classificatory block(the part shown by circles in FIG. 13) is preferably denoted as follows:the upper pixel locating at the i-th from the left and the j-th from thetop in the classificatory block is denoted by B_(ij). Therefore, in theembodiment of FIG. 13, the classificatory block which has an upper pixelB₃₃ as the marked pixel is formed. The lower pixel generated (predicted)from the upper pixels forming the classificatory block (the part shownby a dot in FIG. 13) is preferably denoted by A_(ij) similarly to theupper pixel, hereinafter.

If forming the classificatory block, the classificatory blocking circuit131 outputs it to a classifying circuit 132. The classifying circuit 132classifies the classificatory block into prescribed classes according toits property, and supplies thus obtained class information to aforecasting circuit 134.

To the forecasting circuit 134, a predictive value calculating block isfurther supplied from the blocking circuit for calculation of predictivevalue 133. In the blocking circuit for calculation of predictive value133, for example, 3×3 pixels of predictive value calculating blockcentering the marked pixel B₃₃, as shown by enclosing with a dotted box,is formed and supplied to the forecasting circuit 134.

Here, the method for forming the predictive value calculating block andclassificatory block is not only limited to the above. Note that, theform of the predictive value calculating block is basically free,however, the classificatory block is preferable to include thecharacteristics of the predictive value calculating block.

If receiving the predictive value calculating block and the classinformation of the marked pixel, the forecasting circuit 134 performsadaptive processing to obtain the predicted value of the pixel value ofthe lower pixels by linearly coupling predictive coefficients describedlater corresponding to the received class information to the pixelvalues of the upper pixels forming the predictive value calculatingblock. More specifically, for example, the forecasting circuit 134obtains the predicted values of lower pixels A₄₃, A₄₄, A₄₅, A₅₃, A₅₄,A₅₅, A₆₃, A₆₄ and A₆₅ in the area of 3×3 pixels centering the markedpixel B₃₃ from predictive coefficients corresponding to the class andupper pixels forming the predictive value calculating block B₂₂, B₂₃,B₂₄, B₃₂, B₃₃, B₃₄, B₄₂, B₄₃ and B₄₄.

In the forecasting circuit 134, hereinafter, the similar processing issequentially performed with setting all the upper pixels except for thepixel B₃₃ as the marked pixels. As a result, the predicted values of allthe lower pixels forming the lower hierarchy picture are obtained.

Here, to the forecasting circuit 134, a resolution signal is suppliedfrom a resolution setting circuit 135. The resolution setting circuit135, for example, communicates with the output device to which thepicture is outputted from the adaptive decoding system 7 (in theembodiment of FIG. 1, one of the display devices 9 and 10, printer 11and computer display 12) and recognizing its resolution, and suppliesthe resolution signal representing the recognized resolution to theforecasting circuit 134.

In the forecasting circuit 134, the predictive coefficients of pictureshaving various resolutions have stored, so that a lower hierarchypicture can be obtained using the predictive coefficients of aresolution corresponding to the resolution signal from the resolutionsetting circuit 135.

In addition, it is able to input a resolution to the resolution settingcircuit 135 by operating an operating part (not shown) for example,other than communicating with the output device.

Hereinafter, classifying processing in the classifying circuit 132 andadaptive processing in the forecasting circuit 134 will be described.

First, it will be described about the classifying processing.

As shown in FIG. 14A, now it is assumed that a block composed of 2×2pixels (classificatory block) is formed of a certain marked pixel andthree adjacent pixels, and each pixel is represented by one bit (whichhas either level 0 or level 1). In this case, as shown in FIG. 14B, theblock of 2×2 pixels, four pixels, can be classified into 16 (=(2¹)⁴)patterns based on a level distribution of each pixel. Suchclassification by patterns is the classifying processing.

Note that, the classifying processing can be performed also consideringthe activities (complexity of picture and sudden change) or the like, ofthe picture (picture in the block).

Here, e.g., eight bits degree are normally assigned to each pixel.Moreover, in this embodiment, the classificatory block is composed of5×5 pixels, 25 pixels as described above. Therefore, if the classifyingprocessing is executed on such classificatory blocks, they will beclassified into an enormous number of classes that is (2⁸)²⁵.

To cope with that, the classifying circuit 132 is able to execute theADRC processing on the classificatory blocks previous to the classifyingprocessing. By executing the ADRC processing, the number of bits of thepixels forming a classificatory block can be reduced, and the number ofclasses can be reduced.

Next, the adaptive processing will be described.

For example, it is now considered that the predictive value E[y] of thepixel value (y) of a lower pixel is obtained based on a linear primarycombined model provided by linearly coupling the pixel values of severalupper pixels (hereinafter, they are preferably referred to as learningdata) x₁, x₂, . . . to the prescribed predictive coefficients w₁, w₂, .. . . In this case, the predictive value E[y] can be represented by thefollowing equation:E[y]=w ₁ x ₁ +w ₂ x ₂+ . . .   (1)

To generalize that, a matrix W being a set of the predictivecoefficients (w), a matrix X being a set of the learning data and amatrix Y′ being a set of the predicted values E[y] are defined by thefollowing equations: $X = \begin{pmatrix}x_{11} & x_{12} & \cdots & x_{1n} \\x_{21} & x_{22} & \cdots & x_{2n} \\\cdots & \cdots & \cdots & \cdots \\x_{m1} & x_{m2} & \cdots & x_{mn}\end{pmatrix}$ ${W = \begin{pmatrix}w_{1} \\w_{2} \\\cdots \\w_{n}\end{pmatrix}},{Y^{\prime} = \begin{pmatrix}{E\left\lbrack y_{1} \right\rbrack} \\{E\left\lbrack y_{2} \right\rbrack} \\\cdots \\{E\left\lbrack y_{m} \right\rbrack}\end{pmatrix}}$

As a result, the following observation equation is obtained.XW=Y′  (2)

And it is considered to obtain predicted values E[y] approximate to thepixel values (y) of the lower pixels applying a least square to thisobservation equation. In this case, a matrix Y being a set of the pixelvalues of the lower pixels (hereinafter, they are preferably referred toas teaching data) (y) and a matrix E being a set of the residuals (e) ofthe predicted values E[y] to the pixel values (y) of the lower pixelsare defined by the following equations: ${E = \begin{pmatrix}e_{1} \\e_{2} \\\cdots \\e_{m}\end{pmatrix}},{Y = \begin{pmatrix}y_{1} \\y_{2} \\\cdots \\y_{m}\end{pmatrix}}$

As a result, the following residual equation comes from the equation(2).XW=Y+E  (3)

In this case, predictive coefficients w_(i) to be used to obtain thepredictive values E[y] approximate to the pixel values (y) of the lowerpixels can be obtained by minimizing the square error such as thefollowing: ${\sum\limits_{i = 1}^{m}e_{i}^{2}},$

Thus, it can be said that when the solution by that the above squareerror has differentiated by the predictive coefficient w_(i) becomes 0,that is, the optimum value to obtain the predictive values E[y]approximate to the pixels (y) of the lower pixels is the predictivecoefficients w_(i) satisfying the following equation: $\begin{matrix}{{{e_{1}\frac{\partial e_{1}}{\partial w_{i}}} + {e_{2}\frac{\partial e_{2}}{\partial w_{i}}} + \cdots + {e_{m}\frac{\partial e_{m}}{\partial w_{i}}}} = {0\left( {{i = {1{,2,\ldots}}}\quad,n} \right)}} & (4)\end{matrix}$

First, the equation (3) is differentiated by the predictive coefficientw_(i), and obtaining the following equations: $\begin{matrix}{{\frac{\partial e_{i}}{\partial w_{1}} = x_{i1}},{\frac{\partial e_{i}}{\partial w_{2}} = x_{i2}},\ldots\quad,{\frac{\partial e_{i}}{\partial w_{n}} = {x_{im}\left( {{i = 1},2,\ldots\quad,m} \right)}}} & (5)\end{matrix}$

The equations (4) and (5) give the following equation: $\begin{matrix}{{{\sum\limits_{i = 1}^{m}{e_{i}x_{i1}}} = 0},{{\sum\limits_{i = 1}^{m}{e_{i}x_{i2}}} = 0},\ldots\quad,{{\sum\limits_{i = 1}^{m}{e_{i}x_{in}}} = 0}} & (6)\end{matrix}$

Considering the relationship among the learning data (x), predictivecoefficient (w), teaching data (y) and residual (e) in the residualsequation of the equation (3), the following normal equations can beobtained from the equation (6). $\begin{matrix}\left\{ \begin{matrix}{{{\left( {\sum\limits_{i = 1}^{m}{x_{i1}x_{i1}}} \right)w_{1}} + {\left( {\sum\limits_{i = 1}^{m}{x_{i1}x_{i2}}} \right)w_{2}} + \ldots + {\left( {\sum\limits_{i = 1}^{m}{x_{i1}x_{in}}} \right)w_{n}}} = \left( {\sum\limits_{i = 1}^{m}{x_{i1}y_{i}}} \right)} \\{{{\left( {\sum\limits_{i = 1}^{m}{x_{i2}x_{i1}}} \right)w_{1}} + {\left( {\sum\limits_{i = 1}^{m}{x_{i2}x_{i2}}} \right)w_{2}} + \ldots + {\left( {\sum\limits_{i = 1}^{m}{x_{i2}x_{in}}} \right)w_{n}}} = \left( {\sum\limits_{i = 1}^{m}{x_{i2}y_{i}}} \right)} \\{{{\left( {\sum\limits_{i = 1}^{m}{x_{in}x_{i1}}} \right)w_{1}} + {\left( {\sum\limits_{i = 1}^{m}{x_{in}x_{i2}}} \right)w_{2}} + \ldots + {\left( {\sum\limits_{i = 1}^{m}{x_{in}x_{in}}} \right)w_{n}}} = \left( {\sum\limits_{i = 1}^{m}{x_{in}y_{i}}} \right)}\end{matrix} \right. & (7)\end{matrix}$

The normal equations of the equation (7) can be formed by the samenumber as the number of the predictive coefficients (w) to be obtained.Therefore, the optimum predictive coefficient (w) can be obtained bysolving the equation (7): however, to solve the equation (7), it isneeded that the matrix formed of coefficients concerning with thepredictive coefficient (w) is regular). Note that, to solve the equation(7), e.g., a sweeping method (elimination of Gauss-Jordan) or the likecan be applied.

As the above, in the adaptive processing, the optimum predictivecoefficient (w) is previously obtained for each class, and thepredictive values E[y] approximate to the pixel values (y) of the lowerpixels are obtained using that predictive coefficient (w) by theequation (1). This adaptive processing will be performed in theforecasting circuit 134.

For example, it is assumed that the above learning is now performed onthe classes obtained by the classification of the classificatory blocksformed of 5×5 pixels enclosed with a solid line in FIG. 13, andpredictive coefficients w₁(A₄₃) to w₉(A₄₃), w₁(A⁴⁴) to w₉(A₄₄), w₁(A₄₅)to w₉(A₄₅), w₁(A₅₃) to w₉(A₅₃), w₁(A₅₄) to w₉(A₅₄), w₁(A₅₅) to w₉(A₅₅),w₁(A₆₃) to w₉(A₆₃), w₁(A₆₄) to w₉(A₆₄), w₁(A₆₅) to w₉(A₆₅) arerespectively obtained, to respectively obtain the predictive valuesE[A₄₃], E[A₄₄], E[A₄₅], E[A₅₃], E[A₅₄], E[A₅₅], E[A₆₃], E[A₆₄] andE[A₆₅] of lower pixels in a predictive value calculating block, A₄₃,A₄₄, A₄₅, A₅₃, A₅₄, A₅₅, A₆₃, A₆₄ and A₆₅. In this case, in theforecasting circuit 134, the predictive values E[A₄₃], E[A₄₄], E[A₄₅],E[A₅₃], E[A₅₄], E[A₅₅], E[A₆₃], E[A₆₄] and E[A₆₅] of the HD pixels inthe predictive value calculating block, A₄₃, A₄₄, A₄₅, A₅₃, A₅₄, A₅₅,A₆₃, A₆₄ and A₆₅ are respectively obtained according to the followingequations corresponding to the equation (1). $\begin{matrix}\begin{matrix}{{E\left\lbrack A_{43} \right\rbrack} = {{{w_{1}\left( A_{43} \right)}B_{22}} + {{w_{2}\left( A_{43} \right)}B_{23}} + {{w_{3}\left( A_{43} \right)}B_{24}} +}} \\{{{w_{4}\left( A_{43} \right)}B_{32}} + {{w_{5}\left( A_{43} \right)}B_{33}} + {{w_{6}\left( A_{43} \right)}B_{34}} +} \\{{{w_{7}\left( A_{43} \right)}B_{42}} + {{w_{8}\left( A_{43} \right)}B_{43}} + {{w_{9}\left( A_{43} \right)}B_{44}}} \\{{E\left\lbrack A_{44} \right\rbrack} = {{{w_{1}\left( A_{44} \right)}B_{22}} + {{w_{2}\left( A_{44} \right)}B_{23}} + {{w_{3}\left( A_{44} \right)}B_{24}} +}} \\{{{w_{4}\left( A_{44} \right)}B_{32}} + {{w_{5}\left( A_{44} \right)}B_{33}} + {{w_{6}\left( A_{44} \right)}B_{34}} +} \\{{{w_{7}\left( A_{44} \right)}B_{42}} + {{w_{8}\left( A_{44} \right)}B_{43}} + {{w_{9}\left( A_{44} \right)}B_{44}}} \\{{E\left\lbrack A_{45} \right\rbrack} = {{{w_{1}\left( A_{45} \right)}B_{22}} + {{w_{2}\left( A_{45} \right)}B_{23}} + {{w_{3}\left( A_{45} \right)}B_{24}} +}} \\{{{w_{4}\left( A_{45} \right)}B_{32}} + {{w_{5}\left( A_{45} \right)}B_{33}} + {{w_{6}\left( A_{45} \right)}B_{34}} +} \\{{{w_{7}\left( A_{45} \right)}B_{42}} + {{w_{8}\left( A_{45} \right)}B_{43}} + {{w_{9}\left( A_{45} \right)}B_{44}}} \\{{E\left\lbrack A_{53} \right\rbrack} = {{{w_{1}\left( A_{53} \right)}B_{22}} + {{w_{2}\left( A_{53} \right)}B_{23}} + {{w_{3}\left( A_{53} \right)}B_{24}} +}} \\{{{w_{4}\left( A_{53} \right)}B_{32}} + {{w_{5}\left( A_{53} \right)}B_{33}} + {{w_{6}\left( A_{53} \right)}B_{34}} +} \\{{{w_{7}\left( A_{53} \right)}B_{42}} + {{w_{8}\left( A_{53} \right)}B_{43}} + {{w_{9}\left( A_{53} \right)}B_{44}}} \\{{E\left\lbrack A_{54} \right\rbrack} = {{{w_{1}\left( A_{54} \right)}B_{22}} + {{w_{2}\left( A_{54} \right)}B_{23}} + {{w_{3}\left( A_{54} \right)}B_{24}} +}} \\{{{w_{4}\left( A_{54} \right)}B_{32}} + {{w_{5}\left( A_{54} \right)}B_{33}} + {{w_{6}\left( A_{54} \right)}B_{34}} +} \\{{{w_{7}\left( A_{54} \right)}B_{42}} + {{w_{8}\left( A_{54} \right)}B_{43}} + {{w_{9}\left( A_{54} \right)}B_{44}}} \\{{E\left\lbrack A_{55} \right\rbrack} = {{{w_{1}\left( A_{55} \right)}B_{22}} + {{w_{2}\left( A_{55} \right)}B_{23}} + {{w_{3}\left( A_{55} \right)}B_{24}} +}} \\{{{w_{4}\left( A_{55} \right)}B_{32}} + {{w_{5}\left( A_{55} \right)}B_{33}} + {{w_{6}\left( A_{55} \right)}B_{34}} +} \\{{{w_{7}\left( A_{55} \right)}B_{42}} + {{w_{8}\left( A_{55} \right)}B_{43}} + {{w_{9}\left( A_{55} \right)}B_{44}}} \\{{E\left\lbrack A_{63} \right\rbrack} = {{{w_{1}\left( A_{63} \right)}B_{22}} + {{w_{2}\left( A_{63} \right)}B_{23}} + {{w_{3}\left( A_{63} \right)}B_{24}} +}} \\{{{w_{4}\left( A_{63} \right)}B_{32}} + {{w_{5}\left( A_{63} \right)}B_{33}} + {{w_{6}\left( A_{63} \right)}B_{34}} +} \\{{{w_{7}\left( A_{63} \right)}B_{42}} + {{w_{8}\left( A_{63} \right)}B_{43}} + {{w_{9}\left( A_{63} \right)}B_{44}}} \\{{E\left\lbrack A_{64} \right\rbrack} = {{{w_{1}\left( A_{64} \right)}B_{22}} + {{w_{2}\left( A_{64} \right)}B_{23}} + {{w_{3}\left( A_{64} \right)}B_{24}} +}} \\{{{w_{4}\left( A_{64} \right)}B_{32}} + {{w_{5}\left( A_{64} \right)}B_{33}} + {{w_{6}\left( A_{64} \right)}B_{34}} +} \\{{{w_{7}\left( A_{64} \right)}B_{42}} + {{w_{8}\left( A_{64} \right)}B_{43}} + {{w_{9}\left( A_{64} \right)}B_{44}}} \\{{E\left\lbrack A_{65} \right\rbrack} = {{{w_{1}\left( A_{65} \right)}B_{22}} + {{w_{2}\left( A_{65} \right)}B_{23}} + {{w_{3}\left( A_{65} \right)}B_{24}} +}} \\{{{w_{4}\left( A_{65} \right)}B_{32}} + {{w_{5}\left( A_{65} \right)}B_{33}} + {{w_{6}\left( A_{65} \right)}B_{34}} +} \\{{{w_{7}\left( A_{65} \right)}B_{42}} + {{w_{8}\left( A_{65} \right)}B_{43}} + {{w_{9}\left( A_{65} \right)}B_{44}}}\end{matrix} & (8)\end{matrix}$

Note that, the adaptive processing differs from interpolation processingfrom the viewpoint of that components included in only the lowerhierarchy picture are represented. That is, it can be said that theadaptive processing is the same as the interpolation processing usingso-called interpolation filter, so far as only the equations (1) and (8)are viewed. However, since the predictive coefficients (w) correspondingto the tap coefficients of the interpolation filter are obtained byusing the teaching data (y) quasi learning, also the components includedin the lower hierarchy picture can be represented. From that reason, theadaptive processing can be said that quasi a processing having aresolution creating function.

The detail of the adaptive processing has been disclosed in thepublication of the patent laid-open No. H5(1993)-328185 previously filedby the present applicant, for example.

Now, FIG. 15 shows a structural example of the forecasting circuit 134of FIG. 12.

The class information from the classifying circuit 132 are supplied to aswitch 141. The switch 141 selects one of terminals a1 to a4corresponding to the resolution signal from the resolution settingcircuit 135. The terminals a1 to a4 of the switch 141 are connected tothe address terminals (AD) of read only memories (ROM) 143 to 146respectively. Thereby, the class information from the classifyingcircuit 132 is supplied to one of the ROMs 143 to 146 via the switch 141as an address.

A switch 142 selects one of terminals b1 to b4 corresponding to theresolution signal from the resolution setting circuit 135 similarly tothe switch 141. The terminals b1 to b4 are connected to the dataterminals D of the ROMs 143 to 146 respectively. Note that, the switch142 interlocks the switch 141, and for example, if the switch 141selects one of the terminals a1 to a4, the switch 142 selectscorresponding one of terminals b1 to b4.

In the ROMs 143 to 146, the predictive coefficients for each classobtained by the learning described above have been stored in an addresscorresponding to the class. Specifically, in the ROMs 143 to 146, thepredictive coefficients to be used to convert, for example, if theaforementioned hierarchical coding is performed, the picture of the mostupper hierarchy into a lower hierarchy picture corresponding to theresolution of the display device 9 or 10, printer 11 or computer display12 have been stored respectively.

To an arithmetic circuit 147, the predictive value calculating block issupplied from the blocking circuit for calculation of predictive value133, and the predictive coefficient is supplied from one of the ROMs 143to 146 is supplied via the switch 142, respectively. The arithmeticcircuit 147 performs a sum of products operation corresponding to theequation (1) or (8) using the predictive value calculating block and thepredictive coefficient, and obtaining the lower hierarchy picturecorresponding to the resolution of the output device.

In the forecasting circuit 134 constructed as the above, one of theterminals a1 to a4 is selected by the switch 141 corresponding to theresolution signal from the resolution setting circuit 135, and also acorresponding one of the terminals b1 to b4 is selected by the switch142 with interlocking it.

Then, the class information from the classifying circuit 132 is suppliedto the address terminal of one of the ROMs 143 to 146 that has beenconnected to the terminal selected by the switch 141 (one of theterminals a1 to a4) (hereinafter, it is preferably referred to asselected ROM). In the selected ROM, the predictive coefficient stored inthe address corresponding to the class supplied to its address terminalis read out and outputted from its data terminal.

Since the switch 142 interlocks the switch 141 as described above, thepredictive coefficient read out from the selected ROM is supplied to thearithmetic circuit 147 via the switch 142.

To the arithmetic circuit 147, the predictive value calculating block issupplied from the blocking circuit for calculation of predictive value133 in addition to the predictive coefficient, as described above. Inthe arithmetic circuit 147, the sum of products operation correspondingto the equation (1) or (8) is performed using the predictive valuecalculating block and the predictive coefficient supplied thereto, sothat the lower hierarchy picture corresponding to the resolution of theoutput device is generated and outputted.

Thus, the user can view the picture corresponding to the output device.

Note that, in the above embodiment, the predictive coefficients havestored in the adaptive decoding system 7. However, the predictivecoefficients may be transmitted from the transmitting side as theinformation necessary for decoding. Also methods for forming theclassificatory block or the predictive value calculating block can beinstructed from the transmitting side.

Industrial Capability

The present invention is applicable to a video processing systemincluding a video camera, ATV, video editor, etc., which deals withvideo data having a standard resolution, high resolution and lowresolution.

1. An adaptive decoding system for receiving and decoding a plurality ofvideo signals of different resolutions corresponding to a plurality ofdisplay units having the same resolutions, comprising: an integratedservices digital broadcasting (ISDB) receiver having a signal separatorfor receiving and separating ISDB data into a bit stream encoded withadaptive dynamic range coding (ADRC), additive information, and a timingcode; an additive information decoder for decoding the additiveinformation; an ADRC decoding circuit for demultiplexing and decodingsaid bit stream on a block basis using said time code and forming thedecoded blocks of said bit stream into said plurality of value signalsusing a frame forming circuit; and a selector for selectively outputtingthe decoded plurality of video signals and the decoded additiveinformation to the corresponding plurality of display units having thesame resolution.
 2. The adaptive decoding system according to claim 1,wherein said additive information includes computer graphics and networkdata.
 3. The adaptive decoding system according to claim 1, wherein saidadditive information comprises information of local area interest to aviewer and is identified by an area code.
 4. The adaptive decodingsystem according to claim 1, wherein said plurality of video signalscomprises a high definition video signal and a standard definition videosignal.